後ろ髪を引かれる

うしろがみをひかれる【後ろ髪を引かれる】

(exp) to do something with painful reluctance; (lit: the pull the hairs on one's back)